Method and system for rule-based augmentation of perceptions

ABSTRACT

One embodiment can provide a system for augmenting perceptions of a machine sensing system. During operation, one or more sensors of the system can obtain sensory information associated with a physical system. The system can determine a state of a first component of the physical system based on the obtained sensory information; select, from a rule database, one or more logical rules associated with at least the first component; and augment a perception of the machine sensing system toward the physical system based on both the determined state of the first component and the selected one or more logical rules, thereby facilitating the machine sensing system to make a decision associated with the physical system.

BACKGROUND Field

This disclosure is generally related to machine sensing. Morespecifically, this disclosure is related to A system and method thatimplements logic rules to augment the perception of a machine sensingsystem.

Related Art

Recent developments in machine vision technologies have madecomputer-assisted servicing or remote servicing possible. For example,augmented reality (AR) can be used in the service industry to facilitatea tethered telepresence, a visual retrieval of information, and a repairscript with overlays. In a tethered telepresence, a remote techniciancan interact with a user to provide guided repair. In a visual retrievalof information (e.g., smartphone apps for car manuals), a camera mayretrieve a model number, manual pages, or telemetry statistics. In arepair script with overlays, a checklist or procedural prompt may beoverlaid on a user's view, and the user can click through the overlaidview.

In all the above cases, a user of a local physical system can beequipped with machine vision equipment (e.g., a head-mounted camera orsmartglasses) that can capture images of the local physical system.Information associated with the local physical system (e.g., states ofthe different components within the local physical system) can beinferred based on the captured images. However, visual informationcaptured by the in-field camera can be noisy and intermittent.

SUMMARY

One embodiment can provide a system for augmenting perceptions of amachine sensing system. During operation, one or more sensors of thesystem can obtain sensory information associated with a physical system.The system can determine a state of a first component of the physicalsystem based on the obtained sensory information; select, from a ruledatabase, one or more logical rules associated with at least the firstcomponent; and augment a perception of the machine sensing system towardthe physical system based on both the determined state of the firstcomponent and the selected one or more logical rules, therebyfacilitating the machine sensing system to make a decision associatedwith the physical system.

In a variation on this embodiment, logical rules within the ruledatabase are generated based on domain knowledge associated with thephysical system.

In a variation on this embodiment, a respective selected rule isassociated with both the first component and a second component withinthe physical system, and augmenting the perception comprises determininga state of the second component based on both the state of the firstcomponent and the selected one or more logical rules.

In a variation on this embodiment, a respective selected rule caninclude a persistence rule specifying a predetermined duration, andaugmenting the perception of the machine sensing system can includeallowing the determined state of the first component to persist duringthe predetermined duration.

In a further variation, the persistence rule further specifies arule-overriding condition.

In a variation on this embodiment, the system generates a repair ormaintenance plan for the physical system based on the augmentedperception of the machine sensing system toward the physical system.

In a further variation, the one or more sensors include at least oneimage sensor belonging to an augmented reality system. The systemdisplays instructions for executing the repair or maintenance plan byoverlaying the instructions on images captured by the image sensor.

BRIEF DESCRIPTION OF THE FIGURES

FIG. 1 presents an exemplary use scenario of thelogical-perception-augmentation system, according to one embodiment.

FIG. 2 illustrates operating instructions overlaid onto the image of aprinter, according to one embodiment.

FIG. 3 presents a diagram illustrating an exemplaryperception-augmentation system, according to one embodiment.

FIG. 4 presents a flowchart illustrating an exemplary AR-assisted repairor maintenance process, according to one embodiment.

FIG. 5 illustrates an exemplary computer system that facilitates aperception-augmentation system, according to one embodiment.

In the figures, like reference numerals refer to the same figureelements.

DETAILED DESCRIPTION

The following description is presented to enable any person skilled inthe art to make and use the embodiments, and is provided in the contextof a particular application and its requirements. Various modificationsto the disclosed embodiments will be readily apparent to those skilledin the art, and the general principles defined herein may be applied toother embodiments and applications without departing from the spirit andscope of the present disclosure. Thus, the present invention is notlimited to the embodiments shown, but is to be accorded the widest scopeconsistent with the principles and features disclosed herein.

Overview

The embodiments described herein solve the technical problem ofenhancing perceptions of a machine vision system by incorporatingdomain-specific logical knowledge when extracting information fromcaptured images.

More specifically, the machine system can apply a set of predeterminedlogical rules (which can be based on knowledge of the physical systemunder observation), which can be in the form of frame axioms, to inferand sometimes persist the states of components within the physicalsystem under observation. The machine vision system can include one ormore inference engines and a rule database. The inference engines cancreate an expanded persistent estimate of a state of the physical worldbased on one or more logical rules extracted from the rule database.Because the logical rules are generated based on an extended history ofperceptions received by the machine vision system and the backgrounddomain knowledge of the physical world, the estimated state of thephysical world is consistent with the perception system and thebackground knowledge.

Machine Vision System with Enhanced Perception

Autonomous systems that use sensors to collect information about thephysical world face the problem of making decisions based on signalsthat contain noisy information about quantities in the physical world.For example, sound and light sensors inevitably pick up background soundand light signals, or the global positioning system (GPS) sensors canhave limited precision. Typically, the automated system can usefiltering on perceptual inputs to increase the smoothness andreliability of processes that depend on sensors. For example, Kalmanfilters can be used to smooth the estimates of continuous quantities,such as the coordinates of a tracked object.

In the case of machine sensing systems for the extraction of discretesemantic states (e.g., a machine vision system), instead of returningcontinuous quantities, the sensors return high-level semantic concepts,such as the opened or closed state of a door on a printer or thepresence of a particular component on a device. One cannot apply simplefilters (e.g., Kalman filters) to the machine vision problem. Moreover,machine vision systems for extracting discrete semantic states also faceother challenges, such as occlusion. For example, if the hood of abeing-serviced car is closed, the system cannot obtain information aboutthe state of the engine. Alternatively, if the door for accessing atoner cartridge of a printer is closed, the system cannot obtaininformation about the state of the toner cartridge. In an egocentricvision system where the images are captured by a wearable camera whichmoves along with the user's gaze, the user's viewpoint can changerapidly, meaning that a state of relevant parts of the environmentsometimes is ascertainable from the user's view and sometimes not.

To cope with these complexities associated with the machine visionsystem, some existing machine systems rely on custom-designed programcode to post-process the symbolic outputs of the system. However, suchsolutions are difficult to understand and maintain and requirespecialists who understand computer programming.

Some embodiments of the present invention provide alogical-perception-augmentation system that uses logical rules toaugment the perceptual information. More specifically, thelogical-perception-augmentation system can include a plurality ofinference engines for inferring a state of the physical world usingpre-coded or predetermined logical rules. The logical rules can begenerated based on domain knowledge. For example, if the machine sensingsystem is used for service or maintenance of a printer, the logicalrules can be generated based on prior knowledge of the printer. In onescenario, a machine vision system recognizes a printer toner cartridgebut the printer access door is out of view. Based on the prior knowledgeof the printer, the machine vision system can infer that the state ofthe printer door must be open. Such knowledge can be in the form of athree-dimensional (3D) computer-aided design (CAD) model, and thelogical rules can be in the form of high-level semantics. Depending onthe platform of the application, the logical rules can be implementedaccordingly.

In addition to expanding the inference about the instantaneous state ofan observed system, the logical rules can also be used to infer stateover a time interval. More specifically, the system can have built-in ordefault persistence or frame axioms. Such persistence can be used toinfer state of a component at a later time given observation at a priortime. This knowledge can take into account domain specific expectations.For example, if an observation at a particular time is that theprinter's door is open, the machine vision system can assume that, aftera very small time increment, the printer's door remains open. The dooris unlikely to stay open more than 10 minutes or so as replacing acartridge can be done quickly. If an observation at a particular time isthat a complex device (e.g., a printer) has been disassembled, thesystem can assume that, after a relatively short time interval (e.g., afew minutes), it is highly unlikely that the various components are backtogether and that the complex device may very likely still bedisassemble an hour from now.

FIG. 1 presents an exemplary use scenario of thelogical-perception-augmentation system, according to one embodiment. Inthe example shown in FIG. 1, a user 102 is equipped with an augmentedreality (AR) headset (e.g., the Microsoft HoloLens™ system) 104 and isperforming maintenance on a printer 106. The AR headset is capable ofcapturing and displaying images of a real-word or physical object in areal-world environment or scene. Note that the HoloLens can use an RGB-D(red green blue plus depth) camera plus a Simultaneous Localization andMapping (SLAM) style algorithm to build a model of the room, and allowsa user to overlay computer-generated objects (e.g., labeling or actioninstructions) in the real-world scene.

More specifically, images of printer 106 can be sent by AR headset 104to a server 108 via a network 110. Server 108 can include a cognitiveengine that performs various image-recognition operations, such asfeature extraction and tracking. Based on extracted and trackedfeatures, the cognitive engine in server 108 can determine the state ofone or more components of printer 106. For example, the cognitive enginecan determine whether the toner door is open or closed, or whether thepaper tray has been pulled out. Based on the determined state and a setof predetermined maintenance rules or procedures, a strategic-planningmodule can generate a plan and corresponding operating instructions. Insome embodiments, the operating instructions can be sent from server 108to AR headset 104. AR headset 104 can then overlay the instructions ontothe displayed images of printer 106 (e.g., as annotation), prompting theuser to perform corresponding operations. For example, FIG. 2illustrates operating instructions overlaid onto the image of a printer,according to one embodiment. More particularly, in FIG. 2, thecomputer-generated annotation points the user to the location of a frontpanel and prompts the user to open the front panel. In some embodiments,the computer-generated annotation can also be in the format of ananimation, showing the front panel being opened, thus providing a morestraightforward visual cue to the user.

To be able to provide the user with accurate and timely instructions,the cognitive engine needs to know, with great certainty, the currentstate of the device being serviced. However, the accurate stateinformation may not be readily available. For example, as the user movescloser to the being-serviced physical device or equipment, the field ofview (FOV) of the camera may narrow, resulting in certain componentsdisappearing from the FOV. Moreover, the user may move around, such asturning his head, which can also cause certain objects to disappear fromthe FOV of the camera. However, knowing the state of those components orobjects is important for the strategic-planning module to plan for thenext move.

In some embodiments, to facilitate the strategic-planning module to makedecisions, whenever there is an uncertainty regarding the state of oneor more components in the observed physical device or system, themachine vision system can rely on logical rules to infer the state. Morespecifically, the strategic-planning module can receive inputs from aninference module, which provides an inferred state of the one or morecomponents based on the observation as well as the predetermined logicalrules.

For example, when planning for instructions, the strategic-planningmodule needs to be provided with the state of the toner door of theprinter. However, the toner door is not currently in the FOV of thecamera. On the other hand, the image-analysis result indicates that thetoner is shown in the captured image. Based on this information and theprior knowledge of the printer (i.e., the toner can only be seen whenthe toner door is open), the inference module can infer the state of thetoner door as being open. The inferred state can then be sent to thestrategic-planning module, which plans for the instruction to be sent.In this scenario, instead of overlaying an “open the toner door”instruction, the AR headset may overlay an instruction on the image ofthe toner to prompt the user to remove the toner.

FIG. 3 presents a diagram illustrating an exemplaryperception-augmentation system, according to one embodiment.Perception-augmentation system 300 can be used for facilitating AR-basedremote assistance and maintenance. Perception-augmentation system 300can include an observation module 302, an event-generation module 304,an inference module 306, a rule database 308, a strategic-planningmodule 310, and a plan-delivery module 312.

Observation module 302 can be responsible for obtaining observations ofa physical system by collecting sensor data. The sensor data can beprovided by cameras as well as other types of sensors, such as soundsensors, GPS sensors, motion sensors, etc.

Event-generation module 304 can be responsible for generating eventsbased on outputs of observation module 302. In some embodiments,event-generation module 304 analyzes sensor data to provide a currentstate of the physical system. If the sensor data includes images,event-generation module 304 can perform various image-processingoperations, including feature extraction and feature tracking. In otherwords, event-generation module 304 can be responsible for translatingraw sensor data to high-level events.

Inference module 306 can be responsible for inferring the state of theobserved physical system, including the state of each individualcomponent within the observed physical system, based on events receivedfrom event-generation module 304 as well as logical rules selected fromrule database 308. Note that the state of the observed physical systemmay not be inferred from the events alone, because the observed eventsmay be incomplete or contradictive due to sensor noise. To accuratelyand stably infer the state, inference module 306 needs to combine theevents with one or more logical rules from rule database 308. In otherwords, the logical rules enhance or augment the perception ofperception-augmentation system 300 such that the associated machinevision system can obtain state information of the observed physicalsystem beyond the simple observation. The inferred state can bepersistent unless a later observation overrules the inferred state. Insome embodiments, the inferred state can be associated with a timeparameter (e.g., duration of effectiveness). A domain expert can set thetime parameter associated with a state based on knowledge of thephysical system. For example, it is known that under normal operatingconditions, the paper tray of a printer is closed. Hence, if theinferred state of the paper tray is “closed,” such a state can bepersistent over a predetermined time (e.g., a few seconds).

Rule database 308 stores domain-specific logical rules that can be usedfor inferring the state of the physical system. In some embodiments, thelogical rules can encode detailed knowledge regarding the physicalsystem, including the relative positions among components under defaultoperating conditions.

One type of logical rules is known as an augmentation rule, which can beused to infer the instant state of the physical system. For example, anaugmentation rule can be: “if component A is observed to be in a certainstate, then component B, observation of which is unavailable, must be ina particular state,” or “if components A and B are observed to behave ina certain way, then the physical system must be in a certain state,although observation of other components within the physical system isnot available.”

Another type of logical rules is known as a persistence rule, which canbe used to infer the further state of the physical system based on thecurrent state of the physical system or the current observation. Morespecifically, a persistence rule can define a likely duration a statewill persist. For example, a persistence rule can be “if component A isin state X, component A will remain in state X for time t.” In furtherembodiments, the persistence rule may also specify that certainobservations may override the default. For example, a persistence rulecan be: “if component A is in state X, component A will remain in stateX for time t, unless observation O occurs.” In such a scenario, when theobservation breaks the default state, the state of the component needsto be re-inferred based on the observation and other rules.

The persistence rules provide a simpler way for system users to describethe time-varying states of the various components than a fullprobabilistic model. The simplest persistence rule can be in the formof: absent of specific information about a component A, assume that thestate of component A persists. This rule allows the state of a componentto be inferred based on a previous observation, if a current observationof the component is not available. Moreover, the domain knowledge canalso provide information regarding the default states of components. Forinstance, the document feeder on the top of a printer gets opened whenthe user scans a document but is typically closed, meaning that itsdefault state is closed. A persistence rule that utilizes the defaultstate information can be in the following form: given that the defaultstate of a component A is X, in the absence of specific informationabout component A, the current state of component A persists for Tseconds and then revert to default state X.

Strategic-planning module 310 can be responsible for generating repairor maintenance plans based on the inferred state of the physical system.In some embodiments, strategic-planning module 310 can generate a planbased on the inferred state. The plan may include multiple steps, witheach step being associated with an instruction to the user forperforming a certain action, e.g., opening a toner door, removing thetoner, replacing the toner, and closing the toner door, etc.Strategic-planning module 310 may dynamically update or change the planbased on new observations and newly inferred states. When generatinginstructions according to the repair or maintenance plan,strategic-planning module 310 can also take into account the user'sexpertise. A more detailed instruction can be generated and displayed toa novice user, whereas a simpler instruction can be generated anddisplayed to an expert user.

In some embodiments, when the state of a component is unknown either dueto lack of information or due to conflicting observations,strategic-planning module 310 can generate specificinformation-gathering instructions, which prompt the user to performcertain actions to assist the system in obtaining information that canbe used to infer the state of the component. For example, aninformation-gathering instruction can be “please look at the tonercartridge and see if the locking lever has been released,” possiblyaccompanied by an animation showing where the toner cartridge is.

Plan-delivery module 312 can be responsible for delivering the repair ormaintenance plan to the user. In some embodiments, the repair ormaintenance plan can be delivered to the AR headset worn by the user andbe displayed as step-by-step instructions. In further embodiments, thestep-by-step instructions can be displayed as a computer-generatedannotation overlaying the real-world scene captured by the camera. Morespecifically, the instructions can be in the form of an animationshowing the movements of the corresponding components within thephysical system.

As the user performs the repair or maintenance operation according tothe instructions, observation module 302 continues to make observationsof the physical system, event-generation module 304 generates new eventsbased on the observations, and inference module 306 infers the state ofthe physical system based on the new events and associated logical rulesobtained from rule database 308. Strategic-planning module 310 can thendetermine whether the current repair or maintenance plan needs to beupdated based on the current state of the physical system. If so,strategic-planning module 310 updates the plan and sends the updatedplan to plan-delivery module 312, which delivers the updated plan to theuser.

In some embodiments, the various modules in the machine vision systemcan be implemented in the Soar cognitive engine, which is a first-orderlogic framework that allows for definition of abstract classes of rules.The logical rules stored in the rule database can then be written asfirst-order logic rules, which can use quantified variables over genericobject classes. For example, if there are multiple paper trays in aprinter, the rule about the paper tray can be written as: “for all X, Xis a paper tray, if X is open, then X will remain open for time t unlessobservation O is made.” In such a scenario, the logical rules can beapplied to a class of entities (e.g., paper trays) not just anindividual component (e.g., a particular paper tray).

Moreover, the rule-based perception-augmentation system can bemodularized in such a way that other intelligent agents, in addition tothe strategic-planning module, can access the sensor data or the eventsgenerated by the event-generation module and apply their own logicalrules when making decisions.

In some embodiments, the egocentric vision system may collect transientobservations (e.g., observations last for a brief amount of time due toan object moving in and out of the FOV). To ensure that the transientobservations can be obtained by the inference module or inferenceengine, each event generated by the event-generation module can betimestamped and maintained in an event queue. On the other hand, whenthe inference module requests events in order to infer the current stateof the observed physical system, it can issue a command which specifiesa time instant when the latest state inference was performed. Inresponse, the event generation module can return all events stored inthe queue since the specified time instant based on timestamps of thoseevents. This allows the observation module and the inference module tooperate at different frequencies. Note that the observation moduletypically operates at a higher frequency than that of the inferenceengine. Moreover, because the events are timestamped, each individualclient (e.g., other types of intelligent agent) can request eventssuitable for its particular need, thus allowing multi-clientasynchronous access to the events.

FIG. 4 presents a flowchart illustrating an exemplary AR-assisted repairor maintenance process, according to one embodiment. Prior to startingthe AR-assisted repair or maintenance of a physical system, a domainexpert needs to generate a set of domain-specific logical rules based onthe domain model (operation 402). The domain-specific logical rules caninclude both time-independent augmentation rules as well astime-dependent persistence rules. In some embodiments, the persistencerules can be associated with a time parameter, which specifies theduration of state persistence. A persistence rule can also specify aparticular type of observation that can break the persistent state.

During operation, the system obtains sensor data, which can include liveimages of the physical system (operation 404). The system identifiescomponents associated with the sensor data (e.g., captured images)(operation 406). For example, if the physical system is a printer, themachine vision system needs to identify key components (e.g., paper trayor toner) of the printer. The system can then select one or more logicalrules from the rule database based on the identified components(operation 408). More particularly, the system selects rules that areappropriate for the current situation based on the identifiedcomponents. For example, if the paper tray is identified in the capturedimages, the selection will select rules associated with the paper tray.

Subsequently, the system infers the state of the physical system (e.g.,the state of one or more components) based on both the sensor data(e.g., captured images) and the selected rules (operation 410). Theselogical rules expand the machine vision system's perception toward thephysical system beyond the sensor data in such a way that the state of acomponent not in the image can be inferred or a future state of acomponent can be inferred. Moreover, in situations where the sensor datais noisy, applying the logical rules can filter out the noise in thesensor data. For example, if the FOV of the camera changes rapidlycausing a component to move in and out of the FOV, the state of thecomponent can still be inferred using the persistence rule.

The inferred state of the physical system allows the machine visionsystem to generate a repair or maintenance plan (operation 412). Morespecifically, the plan is generated based on the currently inferredstate and can be dynamically updated based on subsequent observations ofthe physical system. The machine vision system can then deliver anddisplay the repair or maintenance plan to the user (operation 414). Insome embodiments, the repair or maintenance plan can be displayed as ARannotations (e.g., the instructions) overlaid onto the live images ofthe physical system. In further embodiments, the displayed instructionscan be in the format of an animation. The user can then perform theneeded operations (e.g., repair or maintenance procedures) on thephysical system (operation 416).

Exemplary Computer and Communication System

FIG. 5 illustrates an exemplary computer system that facilitates aperception-augmentation system, according to one embodiment. Computersystem 500 includes a processor 502, a memory 504, and a storage device506. Computer system 500 can be coupled to a display device 510, akeyboard 512, a pointing device 514, a camera 516, and can also becoupled via one or more network interfaces to network 508. Storagedevice 506 can store an operating system 518 and aperception-augmentation system 520.

Perception-augmentation system 520 can include instructions, which whenexecuted by computer system 500 can cause computer system 500 to performmethods and/or processes described in this disclosure.Perception-augmentation system 520 can include instructions forobtaining observations (observation module 522), instructions forgenerating events based on observations (event-generation module 524),instructions for accessing logical rule database 540 to obtain logicalrules (rule-obtaining module 526), and instructions for inferring states(inference module 528). If perception-augmentation system 520 is usedfor AR-assisted repair or maintenance, perception-augmentation system520 can further include instructions for planning repair or maintenanceoperations (strategic-planning module 530) and instructions fordelivering the plan to users (plan-delivery module 532).

In general, embodiments of the present invention provide a solution foraugmenting perceptions of a machine sensing system. Although machinevision systems are used as examples throughout this disclosure, thescope of this invention is not limited to machine visions. For example,in addition to enhancing the perception of a machine vision system, thesolution provided by embodiments of the present invention can also beused to enhance the perception of a machine auditory system. Given anaudio recording, a machine-learning module (e.g., a one-dimensionalconvolutional neural network (CNN) can be used to classify the audiorecording into an event. This event can participate in the stateinference the same way as any other event (e.g., an event derived fromcaptured images or videos). For instance, if the bearings on the paperfeeder assembly needed replacing, it might make a grating sound duringoperation. Detecting of such sound can be used to infer this state. Thesame principle can also be used to enhance other types of machinesensory.

The methods and processes described in the detailed description sectioncan be embodied as code and/or data, which can be stored in acomputer-readable storage medium as described above. When a computersystem reads and executes the code and/or data stored on thecomputer-readable storage medium, the computer system performs themethods and processes embodied as data structures and code and storedwithin the computer-readable storage medium.

Furthermore, the methods and processes described above can be includedin hardware modules or apparatus. The hardware modules or apparatus caninclude, but are not limited to, application-specific integrated circuit(ASIC) chips, field-programmable gate arrays (FPGAs), dedicated orshared processors that execute a particular software module or a pieceof code at a particular time, and other programmable-logic devices nowknown or later developed. When the hardware modules or apparatus areactivated, they perform the methods and processes included within them.

The foregoing descriptions of embodiments of the present invention havebeen presented for purposes of illustration and description only. Theyare not intended to be exhaustive or to limit the present invention tothe forms disclosed. Accordingly, many modifications and variations willbe apparent to practitioners skilled in the art. Additionally, the abovedisclosure is not intended to limit the present invention. The scope ofthe present invention is defined by the appended claims.

What is claimed is:
 1. A computer-executed method for augmentingperceptions of a machine sensing system, the method comprising:obtaining, by one or more sensors, sensory information associated with aphysical system; determining a state of a first component of thephysical system based on the obtained sensory information; selecting,from a rule database, one or more logical rules associated with at leastthe first component; and augmenting a perception of the machine sensingsystem toward the physical system based on both the determined state ofthe first component and the selected one or more logical rules, therebyfacilitating the machine sensing system to make a decision associatedwith the physical system.
 2. The method of claim 1, wherein logicalrules within the rule database are generated based on domain knowledgeassociated with the physical system.
 3. The method of claim 1, wherein arespective selected rule is associated with both the first component anda second component within the physical system, and wherein augmentingthe perception comprises determining a state of the second componentbased on both the state of the first component and the selected one ormore logical rules.
 4. The method of claim 1, wherein a respectiveselected rule comprises a persistence rule specifying a predeterminedduration, and wherein augmenting the perception of the machine sensingsystem comprises allowing the determined state of the first component topersist during the predetermined duration.
 5. The method of claim 4,wherein the persistence rule further specifies a rule-overridingcondition.
 6. The method of claim 1, further comprising generating arepair or maintenance plan for the physical system based on theaugmented perception of the machine sensing system toward the physicalsystem.
 7. The method of claim 6, wherein the one or more sensorscomprise at least one image sensor belonging to an augmented realitysystem, and wherein the method further comprises displaying instructionsfor executing the repair or maintenance plan by overlaying theinstructions on the images captured by the image sensor.
 8. Anon-transitory computer-readable storage medium storing instructionsthat when executed by a computer cause the computer to perform a methodfor augmenting perceptions of a machine sensing system, the methodcomprising: obtaining, by one or more sensors, sensory informationassociated with a physical system; determining a state of a firstcomponent of the physical system based on the obtained sensoryinformation; selecting, from a rule database, one or more logical rulesassociated with at least the first component; and augmenting aperception of the machine sensing system toward the physical systembased on both the determined state of the first component and theselected one or more logical rules, thereby facilitating the machinesensing system to make a decision associated with the physical system.9. The non-transitory computer-readable storage medium of claim 8,wherein logical rules within the rule database are generated based ondomain knowledge associated with the physical system.
 10. Thenon-transitory computer-readable storage medium of claim 8, wherein arespective selected rule is associated with both the first component anda second component within the physical system, and wherein augmentingthe perception comprises determining a state of the second componentbased on both the state of the first component and the selected one ormore logical rules.
 11. The non-transitory computer-readable storagemedium of claim 8, wherein a respective selected rule comprises apersistence rule specifying a predetermined duration, and whereinaugmenting the perception of the machine sensing system comprisesallowing the determined state of the first component to persist duringthe predetermined duration.
 12. The non-transitory computer-readablestorage medium of claim 11, wherein the persistence rule furtherspecifies a rule-overriding condition.
 13. The non-transitorycomputer-readable storage medium of claim 8, wherein the method furthercomprises generating a repair or maintenance plan for the physicalsystem based on the augmented perception of the machine sensing systemtoward the physical system.
 14. The non-transitory computer-readablestorage medium of claim 13, wherein the one or more sensors comprises atleast one image sensor belonging to an augmented reality system, andwherein the method further comprises displaying instructions forexecuting the repair or maintenance plan by overlaying the instructionson images captured by the image sensor.
 15. A computer system foraugmenting perceptions of a machine sensing system, the systemcomprising: a processor; and a storage device coupled to the processorand storing instructions which when executed by the processor cause theprocessor to perform a method, wherein the method comprises: obtaining,by one or more sensors, sensory information associated with the physicalsystem; determining a state of a first component of the physical systembased on the obtained sensory information; selecting, from a ruledatabase, one or more logical rules associated with at least the firstcomponent; and augmenting a perception of the machine sensing systemtoward the physical system based on both the determined state of thefirst component and the selected one or more logical rules, therebyfacilitating the machine sensing system to make a decision associatedwith the physical system.
 16. The computer system of claim 15, whereinlogical rules within the rule database are generated based on domainknowledge associated with the physical system.
 17. The computer systemof claim 15, wherein a respective selected rule is associated with boththe first component and a second component within the physical system,and wherein augmenting the perception comprises determining a state ofthe second component based on both the state of the first component andthe selected one or more logical rules.
 18. The computer system of claim15, wherein a respective selected rule comprises a persistence rulespecifying a predetermined duration, wherein augmenting the perceptionof the machine sensing system comprises allowing the determined state ofthe first component to persist during the predetermined duration, andwherein the persistence rule further specifies a rule-overridingcondition.
 19. The computer system of claim 15, wherein the methodfurther comprises generating a repair or maintenance plan for thephysical system based on the augmented perception of the machine sensingsystem toward the physical system.
 20. The computer system of claim 19,wherein the one or more sensors comprise at least one image sensorbelonging to an augmented reality system, and wherein the method furthercomprises displaying instructions for executing the repair ormaintenance plan by overlaying the instructions on images captured bythe image sensor.